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Abstract 

Teachers need different tools for constructing 
virtual realities than do professional program- 
mers ■. Teachers building tutoring environments 
need only and should only provide declarative 
and nonquantitative specification of the applica- 
tion, , as such information is sufficient to build 
powerful prototypes or even products when 
exploited properly. We describe our METU- 
TOR tutor-generation system for sequential- 
action skills , which uses means-ends analysis 
on a teachers s declarative specification of a set 
of actions. METUTOR asks the teacher to 
specify conditions for recommending actions , 
preconditions of actions , and expected and ran- 
dom consequences of actions. METUTOR also 
asks the teacher to associate pictorial and/or 
aural representations with facts , and to specify 
how and when to use them. METUTOR pro- 
vides facilities for automatic resolution of 
interactions and conflicts between media 
objects. We show examples from a firefighting 
tutor and a pilot’s emergency tutor. 

1. Introduction 

Virtual reality can enhance a wide range of educa- 
tional simulations. For instance, military training 
includes many specialized procedural skills for 
emergencies, e.g. firefighting. Broad educational 
application of virtual-reality ideas has been limited 
by the necessary complexity of their implementa- 
tion. Our METUTOR project tries to address this 
problem by providing special tools for computer- 
inexperienced teachers and instructors to develop 
their own virtual-reality tutors for procedural skills. 



An example of our target audience is an instructor 
who trains Navy firefighting -team leaders. 

Computer-inexperienced teachers often have trouble 
giving algorithms for procedural skills, although 
they can specify correct sequences of actions in par- 
ticular circumstances. Furthermore, they usually 
cannot describe numerically or mathematically the 
components of the virtual reality, as this often 
requires sophisticated mathematics. This rules out 
all but minimal three-dimensional modeling, and 
usually requires a discrete representation of time. 
But if teachers can indeed perform the skill they 
wish to teach, they must know the best thing to do 
next in any situation. This "local” knowledge is 
close to the concept of declarative specification in 
languages like Prolog, in which the programmer 
specifies what things must be accomplished but not 
entirely their order [4]. 

So our METUTOR project has devised an approach 
that combines a simple language for specifying state 
descriptions and discrete state changes, a simple 
media-object (bitmap and audio) construction facil- 
ity, and a way to relate state descriptions to media 
objects. Our specification syntax is that of Prolog, 
and our implementation is in Quintus Prolog. Our 
state-change semantics [4] is based on means-ends 
analysis with several of our own additions, and our 
handling of media objects is suggested by tech- 
niques of cartoon animation with several additional 
features. 

Means-ends analysis is a problem-solving technique 
useful for a broad range of human problems, and 
people often seem to use something like it in ever- 
day activity. To automate it, we must identify a set 
of possible discrete actions, and preconditions, 
postconditions, and recommendation circumstances 
for each action. Mean-ends specification of actions 



permits a tutor to comment on the appropriateness 
of student actions in solving a problem, since it 
knows the possible reasons for them. We extend 
basic means-ends analysis with context dependency 
(so the same action has different preconditions or 
results depending on other facts that are true when it 
is applied), and most importantly, constrained ran- 
dom changes to the states to provide an element of 
unanticipated challenge to the student. All this 
teacher-supplied action specification is local infor- 
mation, and for most applications will be intuitive; 
see section 3 for examples. What teacher errors in 
specification do occur, as we have observed in forty 
tutors written using the METUTOR system, are usu- 
ally errors of omission not commission, for which 
we have special debugging tools. 

Means-ends analysis works by repeatedly comparing 
the current state and the goal objectives, and then 
selecting the highest-priority action recommended 
for at least some of the differences. If the highest- 
priority action cannot be done immediately, a recur- 
sive subproblem ("precondition recursion") is 
created with goal the preconditions of the desired 
action. If further differences between current state 
and goal objectives remain after the desired action 
has been performed, a new recursive subproblem 
("postcondition recursion") is created for the current 
state. Although more sophisticated methods of plan- 
ning have been developed in artificial intelligence, 
means-ends is surprisingly broad in applicability and 
robust. For instance, random occurrences can be 
taken in stride because means-ends constantly 
recomputes what to do. Means-ends can exploit 
well the automatic backtracking feature of software 
like Prolog, so even if the teacher's priorities for 
actions are poor, analysis will eventually back up 
and try something else. So means-ends is a good 
choice for tutoring software that computer- 
inexperienced teachers must write. 

Using means-ends analysis means that virtual reali- 
ties constructed with METUTOR are not just pas- 
sive, but can advise the student. While the student 
thinks, the tutor reasons hypothetically to find the 
best sequence of actions to solve the problem from 
the current state. Then if the student picks a subop- 
timal action, the tutor can use 22 domain- 
independent mal-rules (of which more than one 
could apply to the same situation), exploiting this 
hypothetical reasoning plus automatic backtracking, 
to understand what the student is doing. Appropri- 
ate tutoring is tied to each mal-rule. For instance: 
—If the student's action does not change any- 



thing, say so. 

—If the original goal is unsolvable, say so and 
stop. 

-If the student has five times avoided a cer- 
tain best action, give a hint. 

-If the student's action's name is similar to 
the best action, give a hint. 

—If the student’s action is the exact opposite 
of the best action, give a hint. 

-If the student’s action only makes sense by 
misreading the current state slightly, point this 
out. 

-If the student’s action is unnecessary to 
solve the problem, point this out. 

—If the student has returned from a digression, 
point out where and how they digressed. (A 
digression is signalled by an action that, while 
perhaps eventually needed, does not make the 
problem easier to solve.) 

-If the student picks a recommended action, 
but that action is not of the highest priority, 
analyze the justifications for both actions in 
terms of differences in precondition chains in 
the simplest terms. 

Hypothetical reasoning and mal-rules have been 
used in other procedure tutors, e.g. [2], but means- 
ends permits especially general mal-rules. (Not to 
be confused with the student mal-rules, 30 addi- 
tional rules check for teacher errors like misspelled 
words and impossible preconditions.) 

2. Specification of graphics 

A central objective of METUTOR is to help the 
teacher set up a many-to-many mapping between 
facts describing the virtual reality and media objects 
depicting it. That is, a fact or facts will correspond 
to media objects or objects, as in the more general 
Prolog-based approach of [3]. Teachers must obtain 
or construct images and sounds, and we provide 
tools to facilitate this. For audio, the teacher will 
specify a set of frequencies and amplitudes of those 
frequencies as a function of time within a time 
period, which permits a variety of simple noises. 
For images we already provide a simple facility for 
freehand line-drawing using the mouse, plus the 
capability of reusing bitmap pictures from a library 
[5]. Regions can be drawn, moved, scaled, clipped, 
filled, among other standard graphics operations. 
When finished, the screen location of the bitmap 
and its associated fact or facts are associated. Real- 
istic images are not needed in many tutors for adult 
students, since such students can understand meta- 
phoric symbols; otherwise, a professional program - 
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mer could later improve an teacher’s images and 
sounds. Figures 1 and 2 show example metaphori- 
cal displays. 

The mapping between internal facts and media 
objects can be many-to-many when the semantics of 
the two is sufficiently different. For instance, a big 
fire in a compartment on a Navy ship can be shown 
by bitmaps of flames scattered across the picture. 
Conversely, multiple facts may correspond to a sin- 
gle image if some facts are necessary to explain oth- 
ers. For instance, the facts that the oxygen has been 
tested and the oxygen is safe can be displayed in a 
single image, a dial reading "OK". 

An object-oriented system, with inheritance and 
defaults, is the obvious way to assign properties to 
media objects. For instance for a firefighting tutor, 
the instances of flames displayed on the screen can 
inherit shape and color from an object representing 
a single prototypical flame, but could override its 
default size and screen-position attributes when the 
fire is almost out. In general, a media object can 
have these inheritable properties (some of which are 
not yet implemented): 

(1) a pointer to its display representation 
(image or audio); 

(2) a set of immediate generalizations (super- 
concepts); 

(3) size if an image or duration if audio; 

(4) brightness if an image or loudness if audio 
(maximum brightness in our implementation); 

(5) color if an image (each has a single color 
in our implementation); 

(6) location on the screen if an image or start- 
ing time if audio; 

(7) orientation on the screen if an image; 

(8) periodicity information if an intermittent 
image or sound; 

(9) contextual conditions under which it is to 
be displayed (see below); 

(10) overlay plane if an image (see below); 

(1 1) a set of pointers to its subparts. 

And pointers to these objects are stored in a table 
whose retrieval keys are single facts or sets of facts. 

METUTOR provides two ways to specify interac- 
tions between media objects. First, as a simple 
method for handling image overlap, the teacher can 
specify occlusion order for image pairs. For 
instance, the image of the medical corpman could 
always be displayed occluding ("in front of") the 
image of their patient, as in Figure 1. Typically, 
movable objects should occlude setting-depicting 



images, and people should occlude movable objects. 
On the other hand, flames and smoke are more tran- 
sparent, and should not occlude one another even if 
their images overlap, so no occlusion-order 
specification is needed for them. Images can have 
holes in which to place other images, like an outline 
of a tank with room to draw a water level within it. 
(Simultaneous audio objects just have their signals 
added, since sounds do not occlude.) 

The second kind of interaction between media 
objects that we provide for is contextual variation in 
object graphical properties. This is common. For 
instance, only when the firefighters are at the fire 
can they see flames; and then they are shown at the 
center of the screen, otherwise to the right side. As 
another example, if a fire team member gets injured, 
that should reduce by one the number of fire team 
members shown normally. This context dependency 
differs from the oxygen-display example: The con- 
text facts will be displayed normally, but their pres- 
ence causes other facts to be displayed differently. 
In general, the presence or absence of any fact 
could affect the display of another. 

Surprisingly, we discovered when we began imple- 
menting the preceding ideas that some natural- 
language output was essential. To some extent, we 
can avoid it with metaphorical images and sounds, 
like radiating lines over a person's head to indicate 
that they are angry, or sounds of people yelling. 
But this can hurt clarity: Lines over a person’s head 
could mean saintliness or a bad haircut. So we 
always display a text box that completely describes 
the current state. This is essential for currently- 
undisplayed facts (like what is happening at the fire 
when you are not there), orders that are currently in 
effect, descriptions of mental states, history, and so 
on, as well as clarifying displayed information. To 
facilitate natural-language output, we provide a 
modifiable rule-based system for paraphrasing inter- 
nal representations of state descriptors and actions. 
We also let text to be drawn directly on the picture, 
which allows labeling. A fact can be represented by 
multiple pictures and/or multiple texts, like a picture 
of a switch with labels on its positions. 

Our current implementation is in Prolog with 
Prowindows and uses bitmaps to hold all stored 
images. The hardware is Sun Sparc workstations, 
and this does impose performance limitations 
because these machines are not designed for real- 
time graphics. So we use bitmaps for all images 
rather than drawing even simple shapes in real time. 
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Even bitmaps can take a second to load. So a tech- 
nique for additional efficiency that we have explored 
is to recompute certain combinations of bitmaps that 
are commonly shown together, "compiling” them 
into one big bitmap which can be loaded more 
quickly than the individuals. 

The METUTOR framework extends easily to multi- 
student virtual realities, as in replenishment at sea, a 
skill which requires the cooperation of two ship 
commanders [6]. In our implementation of this 
tutor, each student runs a separate program copy 
with its own goals, and each program writes into 
and reads from a shared state file. Each program 
repeatedly checks the state file before changing it; 
whenever the file is changed, the other program 
throws away its previous reasoning and begins 
reanalyzing the situation with the new state. 

3. Examples 

Figure 1 shows an example from the tutor previ- 
ously mentioned for fire team leaders on Navy 
ships, based on the analysis of [7]. The text box is 
on top, the graphical representation of the current 
state is below, and the menu of actions to select is 
in the lower right. Figure 1 shows a tutoring ses- 
sion during which the student ordered watching of 
the fire area for reignition of the now-extinguished 
fire, but he ordered it before emptying the compart- 
ment of smoke, and has thereby caused the reflash 
watchman to collapse from smoke inhalation. So 
the student then directed the medical corpman to 
give first aid to the casualty. In the default color 
implementation, the lower-left shapes are light blue, 
the water is green, the smoke is yellow, the medic 
on the right has a red cross, the other people are 
dark blue, and the other shapes are black. Note the 
oxygen canister occludes the oxygen tester and the 
medical corpman occludes the patient. The very 
bottom of the text box (the description of the 
current state, almost identical to that of the previous 
state described) has been obscured to permit seeing 
some of the previous tutoring. Note that the tutor 
lets students do foolish things like this sending une- 
quipped people into smoke-filled compartments 
because the visual consequences are clearer than 
words, although the tutor does hint. Note also that 
the tutor noticed that the student has returned from a 
digression on their previous action. 

Figure 2 shows an example from the tutor for emer- 
gency fuel problems on F-4 aircraft [1]. It shows a 
session in which the student (a pilot) is doing some- 



thing useful, but not the most important action at 
the moment, so the tutor confines itself to hints. 
Since the name of the best action is similar to that 
the student chose, a lexical confusion may have 
occurred. Just four bitmaps, besides the text, were 
used to create the entire picture; this was deliberate, 
because real cockpits have easily-confusable 
switches. Simple subroutines permitted separating 
the shape and position specifications. 

To illustrate declarative teacher specifications, here 
are examples from the firefighting tutor as they actu- 
ally appear. 

start_state([location(repair, locker), 
raging(fire)^mokey]). 

goal([verified(out(fire)),safe(gases), 
safe(oxygen),not(equipped(team)), 
not(smokey),not(watery), 
not(watched(reflashing)), 
not(unreplaced(casualty)), 
n ot(treated(casualty )), 
not(dead(casualty)), 
debriefed(team), 
deenergized(fire,area)]). 

The first two lines above specify that in the starting 
state the fire team is in their repair locker, the fire is 
raging, and the fire area is smokey. The remaining 
lines specify that the student must achieve the fol- 
lowing: the fire is verified to be out, the gases and 
oxygen in the fire area are safe, the fire team in 
unequipped, the fire area is neither smokey nor full 
of water, no reflash watch is on, no unreplaced or 
untreated or dead casualties are present, and the fire 
area is deenergized. 

recommended([out(fire)], extinguish). 

recommended([not(present(casualty))], 
[present(medical, corpman)], 
direct_medical_corpman). 

The first says that if you want the fire to be out, you 
should extinguish it. The second says that if you 
want a casualty to no longer be present, then if 
there is a medical corpman present, you should 
direct the corpman to handle the casualty. 

precondition(extinguish, 

(location(fire),raging(fire), 

equipped(team),set(boundaries), 

confronted(fire)]). 

This says that to extinguish a fire, you must be at 
the fire, the fire must be raging, the fire team must 
be equipped, the boundaries of the fire must be set, 
and you must be facing the fire. 



4 



deletepostcondition(go(fire), 
l location(repair, locker)]). 
addpostcondition(extinguLsh, 

[out(fire), watery, smokey]). 
addpostconditiont extinguish, 
|not(deenergized(fire,area))], 
(present(casualty),dead(casualty), 
present(crater),raging(fire)J, 

’There is a big explosion!’). 

The first definition says that if you go to the fire, 
you are no longer at the repair locker. The second 
says that when a fire is extinguished, the fire is out, 
the area is watery, and the area is smokey. The last 
definition says that in the special case where the stu- 
dent tries to extinguish when power to the fire area 
is on, a casualty is present and dead, a crater in the 
floor is present, the fire is still raging, and the mes- 
sage "There is a big explosion!" is printed. The last 
definition overrides the second when both apply. 

randchange(extinguish,[ ], 
out(fire),raging(fire),0.3, 

’Fire is still raging.’). 

This says that 30% of the time when a student tries 
to extinguish a fire, they fail. 

bmap(raging(fire),[location(fire)], 

fire2,308,135,red). 

This says that if the fire is raging and the fire team 
is at the fire, the screen should show red flames 
(whose bitmap is in file "fire2") with upper left 
comer of the bitmap at (308,135). (The position 
was automatically computed by the shape- 
construction program when the flames were created 
and situated in a dummy box.) 

opclick(380,6,320,175, 

[location(fire), smokey ],desmoke). 

This says that anytime the student clicks the left 
mouse button in the region 320 pixels wide and 175 
pixel high whose upper left comer is at (380,6), 
while at the same time the simulation has the fire 
team at the fire and the fire area is smokey, that 
click means the student wants to desmoke. 

draw_order(present_casualty, 
presen t_medical_corpman). 
draw_order(treated_casualty, 
present_medical_corpman). 

These say that the image of the medical corpman 
should occlude images of the casualty and the 
stretcher (the latter the "treatment"). 
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